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ABSTRACT 



School choice for poor children is likely to be an issue in 
many state and congressional races. Lawmakers and parents need to pay close 
attention to the quality of academic evaluations of school choice programs. 
Evaluations of the program in Milwaukee (Wisconsin) provide a case in point. 
In 1990, Milwaukee instituted the nation's first publicly funded school 
choice program. Data released in February 1996 about the Milwaukee program 
indicates that the reading scores of choice students in their third and 
fourth years were, on average, from 3 to 5 percentile points higher than 
those of comparable low-income public school students. Mathematics scores 
were, on average, 5 and 12 percentile points higher. This evaluation, 
conducted by J. Greene and P. Petersen was designed to correct for 
methodological errors of previous evaluations. Their results confirm a 
growing collection of evidence that shows that private school vouchers are 
the key to educational success for students who are financially trapped in 
failing inner-city public school systems. (Contains 12 references.) (SLD) 
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chool choice for poor children is likely to be an issue this year in many state and congres- 
sional races. The idea that low-income parents should be able to select the schools — public 
or private — they think best for their children strikes at the very heart of America’s educa- 
tional status quo. Lawmakers and parents therefore need to pay close attention to the quality of 
academic evaluations of school choice programs, including the samples used and methodologies em- 
ployed. Evaluations of the Milwaukee, Wisconsin, school choice experiment are a case in point. 

In 1990, Milwaukee instituted the nation’s first publicly funded school choice program, giving 
low-income parents vouchers to send their children to private schools. In February 1996, data from 
the Milwaukee school choice experiment were made available. Professors Jay P. Greene of the Uni- 
versity of Houston Center for Public Policy and Paul E. Peterson of Harvard University’s Program 
in Education Policy and Governance have conducted a detailed assessment of this information. 1 
Their study, released at a conference this summer, reveals that: 

• The reading scores of choice students in their third and fourth years were, on average, from 3 to 
5 percentile points higher, respectively, than those of comparable low-income public school stu- 
dents. 



• Math scores, on average, were 5 and 12 percentile points higher for the third and fourth years, re- 
spectively. 



The implications of these findings are extremely significant. As Greene and Peterson note, “If 
similar success could be achieved for all minority students nationwide, it could close the gap sepa- 
(j-s rating white and minority test scores by somewhere between one-third and more than one-half. 
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Jay P. Greene and Paul E. Peterson, "The Effectiveness of School Choice in Milwaukee: A Secondary Analysis of 
Data from the Program’s Evaluation," American Political Science Association Panel on the Political Analysis of 
Urban School Systems, August-September 1996. 

Ibid., p. 4. 
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Before the release of the Greene and Peterson study, the Milwaukee school choice program had 
been evaluated each year by the man who designed it: political science professor John Witte of the 
University of Wisconsin at Madison. Witte had concluded that “This school experiment [has] not 
yet led to more effective schools.... Choice creates enormous enthusiasm among parents... but stu- 
dent achievement fails to rise .” 3 Because Witte and his fellow researchers did not release their data 
for secondary analysis by other members of the academic community until this year, their evalu- 
ation was the sole source of information on the test performance of Milwaukee school choice stu- 
dents. Many academic specialists, foundations, and policymakers therefore had concluded that 
while school choice clearly satisfied parents, it was not effective in improving the education of low- 
income central-city students. 

Based on their own examination of the data, however, Greene and Peterson found that the statisti- 
cal model used by Witte and his colleagues failed to replicate the “character” of the MSCP experi- 
ment. The most significant flaw was that Witte’s group compared choice school students with a 
much less disadvantaged cross-section of Milwaukee public school students. Witte had compared 
the choice students to a general sample of public school students, a group which was significantly 
different in several ways. For example, the choice students were more likely to come from house- 
holds headed by poor, minority, single mothers, making any comparison between them and the pub- 
lic school sample misleading. Correcting for these errors, Greene, Peterson, and their colleagues at 
the University of Houston and Harvard designed their own study — a study which confirms a grow- 
ing collection of evidence showing that private school vouchers are the key to educational success 
for students who are financially trapped in failing inner-city public school systems. 

HOW THE HOUSTON-HARVARD STUDY WAS DONE 

Greene and Peterson designed an analysis that would correct for the methodological errors of the 
Witte study. The difference between their conclusions and Witte’s is the result of a scientific meth- 
odology made more rigorous in several ways. Among them: 

• Random assignment of students to the test and control groups. 

The Houston-Harvard analysis compared the choice students with students who applied to the 
choice program but were not admitted. The mandate imposed on the Milwaukee voucher pro- 
gram by the Wisconsin state legislature required Milwaukee’s choice schools, if oversubscribed, 
to admit applicants at random; the result was two randomly selected groups of students: those 
who were chosen to participate in the program and those who were not. The system thus set up a 
unique opportunity for researchers to compare students in the choice program with public school 
students from otherwise eligible families whose parents had made an effort to seek an education 
for them at a private school. This improved “control” for the extent to which families value edu- 
cation and desire to participate in the program meant that any differences in outcome could rea- 
sonably be attributed to the experimental condition of choice. 

• Controlling for grade, year, and school. 

Students seeking to participate in the Milwaukee choice program applied each year for a place 
in a particular grade in a particular school (with spots, as indicated above, to be assigned ran- 
domly if the program was oversubscribed). The evaluation data allowed the Houston-Harvard re- 
searchers to control for both the grade to which a student applied and the year of application. 
Although the specific schools to which students applied were not included in the data, re- 



3 Ibid., p. 1. 

4 Ibid., p. 20. 
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searchers were able to control for that variable by taking into account the ethnicity of applicants. 
Over 80 percent of the choice students attended one of three schools; and virtually all of the stu- 
dents who applied to one of these schools were Hispanic, while virtually all who applied to the 
other two were African American. 5 The Witte study failed to take into account that the selection 
of students was only random by grade, school, and year. 

• Measuring academic effects as cumulative. 

Rather than merely measure the changes in test scores from year to year as Witte did, the Hous- 
ton-Harvard researchers measured the cumulative effect of the choice program on test scores for 
each child for the entire time the child was in the program. Controlling for gender, the test scores 
of choice students were compared with those of “non-selected” students who had applied the 
same year for the same grade and were of the same ethnicity. The effects of choice schools on 
test performance were small during the first and second years; during the third and fourth, how- 
ever, choice students made substantial gains. According to statistical tests, positive results of this 
magnitude would not appear had Milwaukee choice schools had no effect on academic perform- 
ance. One of the most egregious errors of the Witte study was that it controlled for a student’s 
test score in the prior year, in effect controlling for the outcomes of the choice program and incor- 
rectly assuming that students learn at uniform rates. 

• Testing for possible errors due to limited data. 

As Professor Witte states, “In terms of demographic characteristics, non-selected... students 
[and choice students] came from very similar homes.... They were also similar in terms of prior 
achievement scores and parental involvement.” 6 However, the data available to the Houston-Har- 
vard researchers covered only 76.2 percent of the selected students and 58.7 percent of the non- 
selected students. To ensure that these two percentages accurately represented the selected and 
non-selected groups overall, the Houston-Harvard researchers made several additional analyses: 

Family background. Controlling for family background was difficult. Both test and background 
questionnaire data were available for only 36.7 percent of the families of selected students and 
21.8 percent of the families of non-selected students. Even among those parents who did re- 
turn a questionnaire, many did not respond to all of the items. Trying to balance between con- 
trolling for additional factors and preserving the size of the sample, the Houston-Harvard 
team controlled for family income and the mother’s education. Those factors were chosen for 
two reasons: 1) Past academic research shows that these factors strongly affect a child’s educa- 
tional performance, and 2) most parents who returned questionnaires responded to those par- 
ticular questions. Adding these two variables to the analysis decreased the sample size but 
changed the results very little. This confirmed that there were no significant differences in 
family background between those for whom complete background data were available and 
those for whom they were not. 

Prior test scores. Two groups randomly assigned to two categories, as was the case in this 
study, can be assumed to be similar. Thus, the Houston-Harvard study did not control for test 
scores prior to the Milwaukee program. Because of missing data, however, the researchers 
checked and found that at the time these two groups applied to the Milwaukee choice schools, 
the available data showed that they had essentially the same test scores. In fact, those who 
were not selected scored one percentile point higher in both reading and math. Although the 
sample size for this analysis was reduced by the fact that test scores at the time of application 



5 Since the number of white students and other minority students for which information was available was so sparse 
that no reliable results could be obtained, these students were deleted from the analysis. 

6 Greene and Peterson, "The Effectiveness of School Choice in Milwaukee," p. 8. 
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were available only for a limited number of applicants, a statistical analysis was run on the fig- 
ures that were available. With only one exception — the fourth year reading scores, which had 
to be based on only 26 observations — the results controlling for prior test scores were not sub- 
stantially different from those found in the main analysis. This confirmed that there were no 
significant academic differences between those students whose prior test scores were reported 
and those whose scores were not reported. 

Potential "selection effects." The Houston-Harvard study’s main analysis clearly revealed sta- 
tistically significant effects in the third and fourth years of a student’s program. This presents 
two possibilities: 1) Either students benefit in measurable ways from the choice experience 
only after participating in the program for three or more years, or 2) students remain in the 
program for three to four years only if they are benefiting from it. To determine which was the 
case, the Houston-Harvard researchers analyzed the first and second year scores of students 
for whom test results were also available in years three and four. The effects of the first two 
years were no different from the effects for all first and second year students. This would im- 
ply that the positive effects of school choice accumulate over time and are not due to different 
student retention rates. 

OTHER RELEVANT STUDIES 

The Houston-Harvard analysis is more methodologically sound than previous studies of the Mil- 
waukee school choice program. More important, its basic conclusions are supported by a growing 

body of research on the value of private schools among urban poor and minority children. 

• In 1990, the Rand Corporation of Santa Monica, California, examined the academic perform- 
ance of children in New York City’s public and Catholic school systems. The results were star- 
tling: Some 95 percent of Catholic high school students graduated, for example, while only 25 
percent made it out of the public schools. Similarly, 75 percent of Catholic students took the 
Scholastic Aptitude Test, but only 16 percent took it in the public schools. 7 

• A 1982 study headed by education scholar James Coleman of the University of Chicago found 
that students in Catholic schools were one grade level ahead of their public school counterparts 
in mathematics, reading, and vocabulary. 8 

• In 1993, the New York State Department of Education found that Catholic schools with 81 per- 
cent to 100 percent minority composition outscored New York City public schools with the 
same percentage of minority enrollment in Grade 3 reading (+17 percent), Grade 3 mathematics 
(+10 percent), Grade 5 writing (+6 percent), Grade 6 reading (+10 percent), and Grade 6 mathe- 
matics (+1 1 per cent). 9 

• Although the 1993 New York State Department of Education study also found that Catholic and 
public schools have similar percentages of students from troubled families with low incomes, 
Anthony Bryk from the University of Chicago reports that Catholic schools expel far fewer chil- 
dren. 10 

• Derek Neal, a professor of economics at the University of Chicago, in a study conducted for the 
National Bureau of Economic Research in Cambridge, Massachusetts, found that among urban 



7 Sol Stem, "The Invisible Miracle of Catholic Schools," City Journal, Summer 1996, p. 16. 

8 J. S. Coleman, T. Hoffer, and S. Kilgore, High School Achievement : Public , Catholic, & Private Schools 
Compared (New York, N.Y.: Basic Books, 1982). 

9 Stem, "The Invisible Miracle of Catholic Schools," p. 16. 

10 Ibid. 
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blacks and Hispanics nationwide, the probability of high school graduation rises from 62 per- 
cent to 88 percent when a student is moved from a public to a Catholic high school. The prob- 
ability of college graduation for these children rises from 1 1 percent to 27 percent. In addition, 
when compared with their public school counterparts, minority students in urban Catholic 
schools can expect roughly 8 percent higher wages in the future simply because they are more 
likely to complete high school and college. 1 1 

CONCLUSION 

School choice is becoming increasingly popular, and polls show it is most popular among urban 
African Americans — the very group served by a program like Milwaukee’s. Because black parents 
are less satisfied with public schools, they want an alternative. A recent study by the Joint Center 
for Political and Economic Studies, a Washington-based think tank that specializes in research on 
African Americans, found that only 5.7 percent of African Americans rated their local public 
schools “excellent,” compared with 19.9 percent of the general population. Conversely, 16.5 per- 
cent of the black sample rated their schools “poor,” compared with only 7.4 percent of the general 
population. 12 

Contrary to the claims of the education establishment, choice does not leave children of poor fami- 
lies in failing schools. Rather, it is their ticket out. Schools of choice have proven their ability — and 
their desire — to improve the academic outcomes of poor children. Recently, Cardinal John J. 
O’Connor of New York offered to take the lowest performing 5 percent of students in New York 
City’s public schools into New York’s Catholic schools, where they can succeed. This also would be 
a huge financial advantage for taxpayers. New York’s Catholic schools educate students at about a 
third of the cost of public schools. A 5 percent transfer would save New York City over $250 mil- 
lion a year. 13 

The academic success of Milwaukee choice students, as well as other students who have been for- 
tunate enough to escape failing urban schools, is now a matter of record. It is time for state legisla- 
tors and Members of Congress to give other lower-income children the same opportunity. 



1 1 Derek Neal, "The Effect of Catholic Secondary Schooling on Educational Attainment," University of Chicago 
Department of Economics and National Bureau of Economic Research, NBER Working Paper 5353, November 
1995. 

1 2 Katherine McFate, "Social Attitudes," Joint Center for Politcal and Economic Studies 1996 National Opinion Poll, 
April 1996. 

1 3 Stern, "The Invisible Miracle of Catholic Schools," p. 14. 
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